Detecting objects near an autonomous device

ABSTRACT

An example autonomous device is configured to detect objects within a vicinity of the autonomous device. The autonomous device is configured to move along a surface. The autonomous device includes a body, at least one long-range sensor on the body configured for detection in a first field, and at least one short-range sensor on the body. Each short-range sensor is configured for detection in a second field directed towards the surface. The second field is smaller than the first field. Each short-range sensor is configured to output signals based on detection of an object within the second field. A control system is configured to control movement of the autonomous device based, at least in part, on the signals.

TECHNICAL FIELD

This specification relates generally to an autonomous device configuredto detect objects within a vicinity of the autonomous device.

BACKGROUND

Autonomous devices, such as mobile robots, include sensors, such asscanners or three-dimensional (3D) cameras, to detect objects in theirpath or in their vicinity. These sensors have a limited field of view.As a result, autonomous devices may be unable to detect objects in theirimmediate vicinity. For example, sensors on an autonomous device may beunable to detect objects close to the ground and near to the autonomousdevice, particularly at its corners. This can be problematic, especiallyin a manufacturing environment where ground-level objects, such asforklifts, can move into the path of the autonomous device.

SUMMARY

An example autonomous device is configured to detect objects within avicinity of the autonomous device. The autonomous device is configuredto move along a surface. The autonomous device includes a body, at leastone long-range sensor on the body configured for detection in a firstfield, and at least one short-range sensor on the body. Each short-rangesensor is configured for detection in a second field directed towardsthe surface. The second field is smaller than the first field. Eachshort-range sensor is configured to output signals based on detection ofan object within the second field. A control system is configured tocontrol movement of the autonomous device based, at least in part, onthe signals. The autonomous device may include one or more of thefollowing features, either alone or in combination.

The at least one short-range sensor may comprise proximity sensors. Theat least one short-range sensor may comprise near-field sensors. Theautonomous device may be, or include, a mobile robot.

The body may comprise one or more corners. A group of short-rangesensors may be arranged at each corner so that second fields of at leastsome of the short-range sensors in each group overlap at least in part.There may be four or more short-range sensors arranged at each of thefour corners so that second fields of adjacent short-range sensors amongthe four or more short-range sensors overlap at least in part. Eachcorner may comprise an intersection of two edges. Each edge of eachcorner may comprise three short-range sensors. Adjacent ones of thethree short-range sensors may have second fields that overlap at leastin part.

The body may have a circular perimeter. Short-range sensors may bearranged along the circular perimeter so that second fields of at leastsome of the short-range sensors overlap at least in part. The body mayhave a curved perimeter. Short-range sensors may be arranged along thecurved perimeter so that second fields of at least some the short-rangesensors overlap at least in part.

The body may comprise a top part and a bottom part. The bottom part maybe closer to the surface during movement of the autonomous device thanthe top part. Short-range sensors may be located on the body closer tothe top part than to the bottom part. At least one short-range sensormay be located adjacent to the top part.

The at least one short-range sensor on the body may be angled towardsthe surface such that a second field of the at least one short-rangesensor is directed towards the surface. A horizontal plane extends fromthe body at 0°, and the surface is at −90° relative to the horizontalplane. Short-range sensors may be directed towards the surface such thatthe second field of at least some of the short-range sensors is between−1° and −90° relative to the horizontal plane. Short-range sensors maybe directed towards the surface such that the second field of all of theshort-range sensors is between −1° and −90° relative to the horizontalplane.

The at least one short-range sensor may be configured to output signalsin response to detecting the object. The at least one short-range sensormay be configured to use non-visible light to detect the object. The atleast one short-range sensor may be configured to use infrared light todetect the object. The at least one short-range sensor may be configuredto use electromagnetic signals to detect the object. The at least oneshort-range sensor may comprise photoelectric sensors.

Each second field may be 30 centimeters (cm) in diameter at most. Eachsecond field may be 20 centimeters (cm) in diameter at most.

The body may include corners. A group of short-range sensor may bearranged at each of the corners. Adjacent ones of the short-rangesensors may have second fields that overlap at least in part such that,for a corner among the corners, there are no blind spots for theautonomous device in a partial circumference of a circle centered at thecorner. The autonomous device may include a bumper that is comprised ofan elastic material. The bumper may be around at least part of aperimeter of the autonomous device. The at least one short-range sensormay be located underneath the bumper. Sensors may be arranged around atleast part of a perimeter of the body.

Short-range sensors may be directed towards the surface on which thedevice travels such that a second field of each short-range sensorextends at least from 15 centimeters (cm) above the surface to thesurface.

An example autonomous device is configured to detect objects within avicinity of the autonomous device. The autonomous device includes a bodyfor supporting weight of an object, wheels on the body to enable thebody to travel across a surface, and a camera on the body to obtainimages in front of the autonomous device. The camera has first fieldthat extends from the body. Sensors may be disposed along at least partof a perimeter of the body. The sensors may have a second field thatextends from the surface to at least a location below the first field.The autonomous device may be, or include, a mobile robot. The autonomousdevice may include one or more of the following features, either aloneor in combination.

The second field may intersect the first field in part. At least two ofthe sensors that are adjacent to each other may have fields that overlapat least partly. The sensors may be configured to use non-visible lightto detect an object. The sensors may be configured to use infrared lightto detect an object. The sensors may be configured to useelectromagnetic signals to detect an object. The sensors may comprisephotoelectric sensors. The sensors may comprise proximity sensorsconfigured to sense an object within at most 20 centimeters. The sensorsmay comprise proximity sensors configured to sense an object within atmost 30 centimeters.

The body may comprise one or more corners. At least one of the cornersmay be defined by edges that support a group of the sensors. The groupof sensors may have fields that overlap at least in part such that, forthe at least one corner, there are no blind spots for the autonomousdevice in a partial circumference of a circle centered at the at leastone corner. The autonomous device may include a rubber bumper along atleast part of the perimeter. The sensors may be underneath the rubberbumper.

Any two or more of the features described in this specification,including in this summary section, can be combined to formimplementations not specifically described herein.

The systems and processes described herein, or portions thereof, can becontrolled by a computer program product that includes instructions thatare stored on one or more non-transitory machine-readable storage media,and that are executable on one or more processing devices to control(e.g., coordinate) the operations described herein. The systems andprocesses described herein, or portions thereof, can be implemented asan apparatus or method. The systems and processes described herein caninclude one or more processing devices and memory to store executableinstructions to implement various operations.

The details of one or more implementations are set forth in theaccompanying drawings and the description below. Other features andadvantages will be apparent from the description and drawings, and fromthe claims.

DESCRIPTION OF THE DRAWINGS

FIG. 1 is a side view of an example autonomous robot.

FIG. 2 is a side view of the example autonomous robot, which showsranges of long-range sensors included on the robot.

FIG. 3 is a top view of the example autonomous robot, which shows rangesof the long-range sensors included on the robot.

FIG. 4 is a top view of the example autonomous robot, which showsshort-range sensors arranged around parts of the robot.

FIG. 5 is a side view of the example autonomous robot, which the showsshort-range sensors arranged around parts of the robot and their fieldsof view.

FIG. 6 is a top view of the example autonomous robot, which shows theshort-range sensors arranged around parts of the robot and their fieldsof view.

FIG. 7 is a side view of the example autonomous robot, which shows abumper and the short-range sensors underneath or behind the bumper.

Like reference numerals in different figures indicate like elements.

DETAILED DESCRIPTION

Described herein are examples of autonomous devices or vehicles, such asa mobile robot. An example autonomous device (or simply “device”) isconfigured to move along a surface, such as the floor of factory. Theexample device includes a body for supporting the weight of an objectand wheels on the body to enable the body to travel across the surface.The example device includes long-range sensors on the body configuredfor detection in a first field of view (FOV) or simply “field”. Forexample, the device may include a three-dimensional (3D) camera that iscapable of detecting an object within its FOV. The example device alsoincludes short-range sensors on the body. Each short-range sensor may beconfigured for detection in a second FOV that is smaller than, ordifferent from, the FOV of each long-range sensor. The short-rangesensors may include near-field sensors or proximity sensors fordetecting within the second FOV. The second FOV may be directed towardthe surface to enable detection of objects in the immediate vicinity ofthe device. For example, the short-range sensors may be configured todetect objects close to the ground and near to the device, particularlyat its corners. Each short-range sensor may be configured to outputsignals based on—for example, in response to—detection of an objectwithin its FOV. A control system may be configured to control movementof the device based, at least in part, on those signals. The controlsystem may be, or include, one or more processing devices, such as amicroprocessor. The control system can also include computing resourcesdistributed to a remote—for example, a cloud—service and, therefore, thecontrol system need not be on-board the robot. In response to detectionof the object, the control system may take appropriate action, such aschanging the device's path or stopping movement or other operation ofthe robot.

The following description includes values relating to sensor parameters,such as FOV. These values are examples only. Different sensors may havedifferent values, and different devices may use different types,numbers, or configurations of sensors.

An example of an autonomous device is autonomous robot 10 of FIG. 1. Inthis example, autonomous robot 10 is a mobile robot, and is referred tosimply as “robot”. Robot 10 includes a body 12 having wheels 13 toenable robot 10 to travel across a surface 14, such as the floor of afactory or other terrain. Robot 10 also includes a support area 15configured to support the weight of an object. In this example, robot 10may be controlled to transport the object from one location to anotherlocation. Robot 10 includes a sensor configuration of the type describedherein. However, the sensor configuration is not limited to robots ofthis type. Rather, the sensor configuration may be used with anyappropriate type of autonomous device, robot, or vehicle.

In this example, robot 10 includes two types of long-range sensors: athree-dimensional (3D) camera and a light detection and ranging (LIDAR)scanner. However, the robot is not limited to this configuration. Forexample, the robot may include a single long-range sensor or a singletype of long-range sensor. For example, the robot may include more thantwo types of long-range sensors.

Referring to FIG. 2, robot 10 includes 3D camera 16 at a front 17 of therobot. In this example, the front of the robot faces the direction oftravel of the robot. The back of the robot faces terrain that the robothas already traversed. In this example, 3D camera 16 has a FOV 18 of 16°off of horizontal plane 20. The placement of 3D camera 16 is such thatthere is about a 350 millimeter (mm) range 21 before the 3D camera candetect an object proximate to the robot, and about a 410 mm range 22before the object can detect the surface 14 on which it is traveling. Inthis example, the 3D camera has a sensing range 31 of about 1900 mm andcan see about 750 mm above surface 14. Robot 10 also includes a LIDARscanner 24 at its back 25. In this example, the LIDAR scanner ispositioned at a back corner of the robot. The LIDAR scanner isconfigured to detect objects within a sensing plane 26. In this example,the sensing plane is about 200 mm above surface 14. The LIDAR scanner isnot capable of detecting objects less than 200 mm above surface 14. Asimilar LIDAR scanner is included at the diagonally opposite frontcorner of the robot, which has the same scanning range and limitations.

FIG. 3 is a top view of robot 10. LIDAR scanners 24 and 23 are locatedat back corner 28 and at front corner 27, respectively. In this example,each LIDAR scanner has a scanning range 29 of about 1000 mm over an arcof about 270°. In some implementations, each LIDAR scanner may have ascanning range of about 12,000 mm over an arc of about 270°. As shown inFIG. 3, the range 31 of 3D camera 16 is about 1900 mm over an arc 33 ofabout 56°. However, after a plane 34, the field of view of 3D camera 16decreases from about 1400 mm to about 1000 mm at the maximum range ofthe 3D camera.

As is evident from FIGS. 2 and 3, in this example configuration, robot10 includes several blinds spots, including at corners 27 and 28. Inthis example, a blind spot includes an area that is not visible to thelong-range sensors. As a result, the long-range sensors cannotaccurately detect objects within that area. For example, robot 10 cannotaccurately detect objects that are less than 200 mm above surface 14.For example, robot 10 cannot accurately detect objects that are lessthan 350 mm from its front 17. Accordingly, short-range sensors areincorporated into the robot to sense in the areas that cannot be sensedby the long-range sensors. Thus, the short-range sensors are able todetect objects that would otherwise go undetected.

In some implementations, each short-range sensor is a member of a groupof short-range sensors that is arranged around, or adjacent to, eachcorner of the robot. The FOVs of at least some of the short-rangesensors in each group overlap in whole or in part to providesubstantially consistent sensor coverage in areas near the robot thatare not visible by the long-range sensors. In some cases, completeoverlap of the FOVs of some short range sensors may provide sensingredundancy.

In the example of FIG. 4, robot 10 includes four corners 27, 28, 35, and36. In some implementations, there are two or more short-range sensorsarranged at each of the four corners so that FOVs of adjacentshort-range sensors overlap in part. For example, there may be twoshort-range sensors arranged at each corner; there may be threeshort-range sensors arranged at each corner; there may be fourshort-range sensors arranged at each corner; there may be fiveshort-range sensors arranged at each corner; there may be sixshort-range sensors arranged at each corner; there may be sevenshort-range sensors arranged at each corner; there may be eightshort-range sensors arranged at each corner, and so forth. In theexample of FIG. 4, there are six short-range sensors 38 arranged at eachof the four corners so that FOVs of adjacent short-range sensors overlapin part. In this example, each corner comprises an intersection of twoedges. Each edge of each corner includes three short-range sensorsarranged in series. Adjacent short-range sensors have FOVs that overlapin part. At least some of the overlap may be at the corners so thatthere are no blind spots for the mobile device in a partialcircumference of a circle centered at each corner.

FIGS. 1, 4, 5, and 6 show different views of the example sensorconfiguration of robot 10. As explained above, in this example, thereare six short-range sensors 38 arranged around each of the four corners27, 28, 35, and 36 of robot 10. As shown in FIGS. 5 and 6, FOVs 40 and41 of adjacent short-range sensors 42 and 43 overlap in part to coverall, some, or portions of blind spots on the robot that are outside—forexample, below—the FOVs of the long-range sensors.

Referring to FIG. 5, the short-range sensors 38 are arranged so thattheir FOVs are directed at least partly towards surface 14 on which therobot travels. In an example, assume that horizontal plane 44 extendingfrom body 12 is at 0° and that the direction towards surface 14 is at−90° relative to horizontal plane 44. The short-range sensors 38 may bedirected (e.g., pointed) toward surface 14 such that the FOVs of all, orof at least some, of the short-range sensors are in a range between −1°and −90° relative to horizontal plane 44. For example, the short-rangesensors may be angled downward between −1° and −90° relative tohorizontal plane 44 so that their FOVs extend across the surface inareas near to the robot, as shown in FIGS. 5 and 6. The FOVs of theadjacent short-range sensors overlap partly. This is depicted in FIGS. 5and 6, which show adjacent short-range sensor FOVs overlapping in areas,such as area 45 of FIG. 6, to create combined FOVs that cover theentirety of the front 17 of the robot, the entirety of the back 25 ofthe robot, and parts of sides 46 and 477 of the robot. In someimplementations, the short-ranges sensors may be arranged to combineFOVs that cover the entirety of sides 46 and 47.

In some implementations, the FOVs of individual short-range sensorscover areas on surface 14 having, at most, a diameter of 10 centimeters(cm), a diameter of 20 cm, a diameter of 30 cm, a diameter of 40 cm, ora diameter of 50 cm, for example. In some examples, each short-rangesensor may have a sensing range of at least 200 mm; however, otherexamples may have different sensing ranges.

In some implementations, the short-range sensors are, or include,time-of-flight (ToF) laser-ranging modules, an example of which is theVL53L0X manufactured by STMicroelectronics®. This particular sensor isbased on a 940 nanometer (nm) “class 1” laser and receiver. However,other types of short-range sensors may be used in place of, or inaddition to, this type of sensor. In some implementations, theshort-range sensors may be of the same type or of different types.Likewise, each group of short-range sensors—for example, at each cornerof the robot—may have the same composition of sensors or differentcompositions of sensors. One or more short-range sensors may beconfigured to use non-visible light, such as laser light, to detect anobject. One or more short-range sensors may be configured to useinfrared light to detect the object. One or more short-range sensors maybe configured to use electromagnetic signals to detect the object. Oneor more short-range sensors may be, or include, photoelectric sensors todetect the object. One or more short-range sensors may be, or include,appropriately-configured 3D cameras to detect the object. In someimplementations, combinations of two or more of the preceding types ofsensors may be used on the same robot. The short-range sensors on therobot may be configured to output one or more signals in response todetecting an object.

Signals from the short-range sensors and from the long-range sensors maybe processed by a control system, such as a computing system, toidentify an object near to, or in the path of, the robot. If necessary,navigational corrections to the path of the robot may be made, and therobot's movement system may be controlled based on those corrections Thecontrol system may be local. For example, the control system may includean on-board computing system located on the robot itself. The controlsystem may be remote. For example, the control system may be a computingsystem external to the robot. In this example, signals and commands maybe exchanged wirelessly to control operation of the robot. Examples ofcontrol systems that may be used are described herein and may includeone or more processing devices, such as a microprocessor, and memorystoring instructions that are executable by the microprocessor tointerpret data based on signals from sensors, to determine navigationalpaths of the robot based on those signals, and to control the movementof the robot based on the determined navigational paths.

The short-range sensors are not limited to placement at the corners ofthe robot. For example, the sensors may be distributed around the entireperimeter of the robot. For example, in example robots that havecircular or other non-rectangular bodies, the short-range sensors may bedistributed around the circular or non-rectangular perimeter and spacedat regular or irregular distances from each other in order to achieveoverlapping FOV coverage of the type described herein. Likewise, theshort-range sensors may be at any appropriate locations—for example,elevations—relative to the surface on which the robot travels.

Referring to FIG. 1 for example, body 12 includes a top part 50 and abottom part 51. The bottom part is closer to surface 14 during movementof the robot than is the top part. The short-range sensors may belocated on the body closer to the top part than to the bottom part. Theshort-range sensors may be located on the body closer to the bottom partthan to the top part. The short-range sensors may be located on the bodysuch that a second field of each short-range sensor extends at leastfrom 15 centimeters (cm) above the surface down to the surface. Thelocation of the short-range sensors may be based, at least in part, onthe FOVs of the sensors. In some implementations, all of the sensors maybe located at the same elevation relative to the surface on which therobot travels. In some implementations, some of the sensors may belocated at different elevations relative to the surface on which therobot travels. For example, sensors having different FOVs may beappropriately located relative to the surface to enable coverage ofblinds spots near to the surface.

In some implementations, such as that shown in FIG. 7, robot 10 mayinclude a bumper 52. The bumper may be a shock absorber and may beelastic, at least partially. The short-range sensors 38 may be locatedbehind or underneath the bumper. In some implementations, theshort-range sensors may be located underneath structures on the robotthat are hard and, therefore, protective.

In some implementations, the direction that the short-range sensorspoint may be changed via the control system. For example, in someimplementations, the short-range sensors may be mounted on body 12 forpivotal motion, translational motion, rotational motion, or acombination thereof. The control system may output signals to the robotto position or to reposition the short-range sensors, as desired. Forexample if one short-range sensor fails, the other short-range sensorsmay be reconfigured to cover the FOV previously covered by the failedshort-range sensor. In some implementations, the FOVs of the short-rangesensors and of the long-range sensors may intersect in part to providethorough coverage in the vicinity of the robot.

The dimensions and sensor ranges presented herein are for illustrationonly. Other types of autonomous devices may have different numbers,types, or both numbers and types of sensors than those presented herein.Other types of autonomous devices may have different sensor ranges thatcause blind spots that are located at different positions relative tothe robot or that have different dimension than those presented. Theshort-range sensors described herein may be arranged to accommodatethese blind spots.

The example robot described herein may include, and/or be controlledusing, a control system comprised of one or more computer systemscomprising hardware or a combination of hardware and software. Forexample, a robot may include various controllers and/or processingdevices located at various points in the system to control operation ofits elements. A central computer may coordinate operation among thevarious controllers or processing devices. The central computer,controllers, and processing devices may execute various softwareroutines to effect control and coordination of the various automatedelements.

The example robot described herein can be controlled, at least in part,using one or more computer program products, e.g., one or more computerprogram tangibly embodied in one or more information carriers, such asone or more non-transitory machine-readable media, for execution by, orto control the operation of, one or more data processing apparatus,e.g., a programmable processor, a computer, multiple computers, and/orprogrammable logic components.

A computer program can be written in any form of programming language,including compiled or interpreted languages, and it can be deployed inany form, including as a stand-alone program or as a module, component,subroutine, or other unit suitable for use in a computing environment. Acomputer program can be deployed to be executed on one computer or onmultiple computers at one site or distributed across multiple sites andinterconnected by a network.

Actions associated with implementing at least part of the robot can beperformed by one or more programmable processors executing one or morecomputer programs to perform the functions described herein. At leastpart of the robot can be implemented using special purpose logiccircuitry, e.g., an FPGA (field programmable gate array) and/or an ASIC(application-specific integrated circuit).

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read-only storagearea or a random access storage area or both. Elements of a computer(including a server) include one or more processors for executinginstructions and one or more storage area devices for storinginstructions and data. Generally, a computer will also include, or beoperatively coupled to receive data from, or transfer data to, or both,one or more machine-readable storage media, such as mass storage devicesfor storing data, e.g., magnetic, magneto-optical disks, or opticaldisks. Machine-readable storage media suitable for embodying computerprogram instructions and data include all forms of non-volatile storagearea, including by way of example, semiconductor storage area devices,e.g., EPROM, EEPROM, and flash storage area devices; magnetic disks,e.g., internal hard disks or removable disks; magneto-optical disks; andCD-ROM and DVD-ROM disks.

Any connection involving electrical circuitry that allows signals toflow, unless stated otherwise, is an electrical connection and notnecessarily a direct physical connection regardless of whether the word“electrical” is used to modify “connection”.

Elements of different implementations described herein may be combinedto form other embodiments not specifically set forth above. Elements maybe left out of the structures described herein without adverselyaffecting their operation. Furthermore, various separate elements may becombined into one or more individual elements to perform the functionsdescribed herein.

What is claimed is: 1-24. (canceled)
 25. An autonomous devicecomprising: a body for supporting weight of an object; wheels on thebody to enable the body to travel across a surface; a camera on the bodyto obtain images in front of the autonomous device, the camera havingfirst field that extends from the body; and sensors disposed along atleast part of a perimeter of the body, the sensors having a second fieldthat extends from the surface to at least a location below the firstfield.
 26. The autonomous device of claim 25, wherein the second fieldintersects the first field.
 27. The autonomous device of claim 25,wherein at least two of the sensors that are adjacent to each other havefields that overlap at least partly.
 28. The autonomous device of claim25, wherein the sensors are configured to use non-visible light todetect an object.
 29. The autonomous device of claim 25, wherein thesensors are configured to use infrared light to detect an object. 30.The autonomous device of claim 25, wherein the sensors are configured touse electromagnetic signals to detect an object.
 31. The autonomousdevice of claim 25, wherein the sensors comprise photoelectric sensors.32. The autonomous device of claim 25, wherein the body comprises one ormore corners, at least one of the corners being defined by edges thatsupport a group of the sensors, the group of sensors having fields thatoverlap at least in part such that, for the at least one corner, thereare no blind spots for the autonomous device in a partial circumferenceof a circle centered at the at least one corner.
 33. The autonomousdevice of claim 32, further comprising a rubber bumper along at leastpart of the perimeter, the sensors being underneath the rubber bumper.34. The autonomous device of claim 25, wherein the sensors compriseproximity sensors configured to sense an object within at most 20centimeters.
 35. The autonomous device of claim 25, wherein the sensorscomprise proximity sensors configured to sense an object within at most30 centimeters.
 36. The autonomous device 25, wherein the autonomousdevice comprises a mobile robot.